Boosting the Throughput of a Multi-server System via Adaptive Task Replication

The throughput (rate of task completion) of a multi-server system is typically the sum of the service rates of individual servers. In this paper we demonstrate how task replication can result in a higher throughput, and we seek to find the fundamental limit of this throughput boost. Several recent works have studied queueing systems where tasks are replicated at multiple servers, and the copies are canceled when any one replica is served. Replication affects the system in two opposing ways: 1) replicas provide load-balancing by finding the shortest among the queues that they join, and 2) redundant time spent by multiple servers on the same task can add load to the system. However, [1–4] observe scenarios where replication can in fact reduce the system load, and boost its throughput. Understanding this effect of simultaneous service of replicas on the throughput is an open problem, which we address in this work. We present a Markov Decision Process (MDP) framework to find the throughput-optimal replication policy. The MDP is hard to solve in general, and we have to resort to myopic replication policies. To help quantify the gap from optimality, we give a bound on the optimal throughput for the two server case.